Sample and descriptive outline

Sample Size

Report total sample size, sample size by visit.

The current data consists of 132 participants; the participants contributed at various time-points as reported in the Table 1 and 2.

Table 1: Initial sample size per timepoint
age N
5 88
10 112
14 106
24 102
36 89

Table 2:N of participants that participanted to 1-5 number of visits at timepoints from 5 to 36 months
N of Visits N
1 9
2 13
3 19
4 50
5 41

Retired participants

3 participants dropped out from the study and were therefore excluded from the analysis, with a final sample size of 129 infants, of whom 100 with familiar risk of ASD.

The data presented hereafter was taken from 500 ms after stimulus onset until 5 seconds (end of trial).

Missing Data

Report percentage of missing data per trial, exclusion criteria and number of excluded participant plus final sample size and data quality per visit

Table 3: Average percentage of missing data, standard deviation, upper and lower quartile per age group across trials for the whole sample
age Mean % SD 75% P 25% P
5 26.72 24.05 42.05 5.91
10 27.11 23.72 41.63 7.42
14 22.88 23.13 38.18 2.42
24 22.12 23.41 34.70 1.89
36 20.00 21.01 32.23 1.97

Table 4: Final sample size per timepoint after the exclusion of participants with 100% of trials with percentage of missing data > 25%
age N
5 80
10 99
14 96
24 96
36 82

The final sample size consisted of 128, of whom 99 with familiar risk of ASD

Table 5: Average percentage of missing data, standard deviation, upper and lower quartile per age group across trials for the final clean sample
age Mean % SD 75% P 25% P
5 9.25 7.84 15.98 1.97
10 9.34 7.97 16.36 1.36
14 7.15 7.56 12.54 0.76
24 7.47 7.84 13.64 0.76
36 7.60 7.62 14.09 1.06

Non-looking trials

Report percentage of non-looking trials and number of excluded participants (if any)

Figure 1: Average Percentage of Non-Looking Trials per participants across trials

Figure 1: Average Percentage of Non-Looking Trials per participants across trials

Descriptive statistics

Report fundamental sample characteristics

Table 5: number of participants pertaining to the different likelihood groups (high-likelihood of ASD (H-L ASD), ADHD (H-L ADHD), both (H-L ASD & ADHD), low-likelihood (L-L), not available (NA))
age N.FEM N H-L ASD N H-L ADHD N H-L ASD & ADHD N L-L N NA N TOT
5 32 44 0 12 24 0 80
10 46 63 0 15 21 0 99
14 44 58 0 16 22 0 96
24 46 59 0 15 22 0 96
36 41 55 0 11 16 0 82
Figure 2: N of Males and Females in age and likelihood groups

Figure 2: N of Males and Females in age and likelihood groups

Raw data plots

Average proportional looking time

Figure 3: Boxplots of the average Proportional Looking Time by age collapsed on time across the trial

Figure 3: Boxplots of the average Proportional Looking Time by age collapsed on time across the trial

Time-series of proportional looking time

Figure 4: Average Proportional Looking Time by age and time in the trial with 95% confidence intervals

Figure 4: Average Proportional Looking Time by age and time in the trial with 95% confidence intervals

Analysis

We aggregated the cleaned data in time bins of 0.5 second and further analysed the proportional looking time data towards the face and the car.

The visual inspection of the proportional-looking time across the duration of the trial suggested a curvature in the rate of change across time; therefore, we decided to include orthogonal polynomials as fixed and random predictors in a multilevel regression model. Additionally, we included age in the model as continuous, fixed predictor; since the measurements are repeated within participants, we included random intercept and slope per participant in the model.

Adding polynomials to the model means adding the value of time raised to the nth power; in this case, we calculated the “orthogonal” – or transformed – polynomials, to avoid intercorrelation between the values. These polynomials help to fit the model to the actual shape of the data, since they describe various curvilinear profiles, as shown in Figure 3 and 4.

Since an increasing polynomial order tend to increase the fit to the data, we planned to select the best fit curve based on meaningful considerations. In this case, when we examine the proportional looking time to one AOI, we may expect at least 1 peak in the probability distribution, described, in polynomials term, as a roughly symmetrical rise and fall – configured by the quadratic polynomial. This configuration may represent a typical response to the experimental design, where the participant’s attention alternates between a target and numerous distractors. Nonetheless, the current design may also allow to go back and forth between target and distractors, therefore, we may expect an asymmetrical peak with 3 changes of focus – from target to distractor, and back to target (even though, in the raw data, we only observe single peaks).

Figure 5: sample curves representing polynomial orders up to degree 3

Figure 5: sample curves representing polynomial orders up to degree 3

For the selection of the ppolynomial order, we split the dataset into a train and test set (respectively, 75% and 25% of the data). We compared the Root Squared Mean Error (RSME) of the model fit to the train and the test data. After selecting the model with the lowest RSME, we tested the contribution of the image display as random intercept, and age as a fixed predictor with the Likelihood Ratio Test (LRT).

In a second round of selection, we restricted the dataset to infants with likelihood information and we tested the contribution of specific likelihood of atypical neuro-development (ASD) as a covariate and in interaction with age.

After selecting the model with the best fit, we calculated the p-values of the beta coefficients with multiple t-tests using Satterthwaite’s approximation method. We calculated the 95% confidence intervals of the beta coefficients with the Wald approximation method, and we tested the individuals contrasts with type III Analysis of Variance.

To further compensate the approximation used for the calculation of p-value and confidence interval, we calculated a prediction interval for each individual age group, including the variation embedded in the fixed effects and the residuals, as well as the conditional modes of the random effects. The prediction interval is illustrated in Figure 5. Finally, we examined the relationship between the individual coefficients, the individual standard errors and the number of data points – and its potential impact on the estimation of the beta coefficients.

Results

Model Selection

Face

We obtained a smaller RSME by adding orthogonal polynomials up to degree 3 with the training set. The model with 3rd degree polynomials with the test set did not obatin a bigger RSME (Table 6).

Table 6: RSME calculated for each model, respectively with the test and training sets.
Polynomial Order Train Set Test Set
- 0.4156 NA
2nd 0.4144 NA
3rd 0.4131 0.4166

The LRT revealed the model including age as fixed predictor as the best fit model (see Table 7). The Chi-Squared degrees of freedom show that this model contains 5 more levels than the base model, as Age contains 5 levels of variation (5, 10, 14, 24 and 36 months). The two models are considerably different from each other and the second model shows a strikingly better fit – which is already a demonstration that Age plays a role in determining the proportional looking time at the face in this setting. All the relevant indicators of the fit of the model (AIC, BIC and logLik) significantly decrease when age is added, showing that this model significantly contributes to explain the variance in the data.

Table 7: LRT between the base model of 3rd polynomial order (no fixed effect), and the model including age as a fixed predictor
term df AIC BIC logLik deviance statistic Chi.Df p.value Sign
m_3_t 16 41449.15 41585.87 -20708.57 41417.15 NA NA NA NA
m_3_t_a 20 40871.93 41042.83 -20415.96 40831.93 585.221 4 0 **

Gender and Likelihood Group

A model with Gender in interaction with Age presented an improvement of the fit:

Table 9: xxx
Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
m_3_t_a_s 20 40871.92 41042.83 -20415.96 40831.92 NA NA NA
m_3_t_a_sex 21 40868.39 41047.84 -20413.19 40826.39 5.538053 1 0.0186073
m_3_t_a_sex_int 28 40853.05 41092.31 -20398.52 40797.05 29.338384 7 0.0001255

Also, after adding sex in interaction with age, the likelihood group turned out to marginally improve the fit of the model:

Table 10: xxx
term df AIC BIC logLik deviance statistic Chi.Df p.value Sign
m_3_t_a_comp 28 40853.05 41092.31 -20398.52 40797.05 NA NA NA NA
m_3_t_a_rg1_comp 29 40855.04 41102.86 -20398.52 40797.04 0.005 1 0.945 -
m_3_t_a_rg2_comp 44 40868.77 41244.76 -20390.38 40780.77 16.275 15 0.364 -

Car

For the AOI car, we adopted a polynomial order of degree 2 by comparing RSMEs across order and the train and the test set (see Table 9). The RSME of the test set increased, suggesting that a 3rd degree polynomial order was overfitting the data.

Table 11: RSME calculated for each model, respectively with the test and training sets.
Polynomial Order Train Set Test Set
- 0.3113 NA
2nd 0.3105 NA
3rd 0.3089 0.309

The model including age as a fixed predictor fitted best to the whole dataset. Furthermore, the model including the interaction between age and likelihood group fitted best the dataset including the infants with likelihood information (see Table 10 and 11).

Table 12: LRT of the best fit model vs the models including age
term df AIC BIC logLik deviance statistic Chi.Df p.value Sign
mc_3_it 16 19767.15 19903.87 -9867.575 19735.15 NA NA NA NA
mc_3_t_a 20 19519.71 19690.61 -9739.855 19479.71 255.44 4 0 **
Tabel 13: LRT of the best fit model vs the models including Likelihood group for ASD
term df AIC BIC logLik deviance statistic Chi.Df p.value Sign
mc_3_t_a_comp 20 19519.71 19690.61 -9739.855 19479.71 NA NA NA NA
mc_3_t_a_rg1_comp 21 19521.60 19701.05 -9739.800 19479.60 0.109 1 0.741 -
mc_3_t_a_rg2_comp 28 19528.17 19767.44 -9736.086 19472.17 7.428 7 0.386 -

Fixed Effects

Face

Table 12 shows the beta coefficients of all the predictors contained in the model. The intercept corresponds to the average height of the curve across the trial, or the average probability for the infants to hit the face irrespective of other predictors. The intercepts stands significantly above the chance level (i.e., the possibility of hitting on the AOI face by chance, corresponding to 1/the number of AOIs, 5). Also, we observe a significant decrease of this value across the trial (i.e., negative slope), irrespective of age. With age, the average height of the curve tend to decrease. The effect of age on the 2nd and 3rd degree polynomials is significant: they respectively tend to become more positive and more negative. A more positive quadratic term with age indicates an increasingly steep decrease of face looking time across the trail with age; a more negative cubic term indicates a higher tendency to peak post-decrease (see the turquoise curve on the right in figure 5). Figure 6 easily illustrate this trend, where the stable preference for the face seems to rapidly decrease over the course of the trial with age.

Table 14: Estimates of fixed effects predicting the proportional looking time to the face, with the 95% confidence intervals
term Estimate st.error df t.value p.value CI_2.5 CI_97.5 Sign
(Intercept) 45.90 1.90 11.50 24.11 0.00 42.17 49.64 **
ot1 -16.32 2.26 434.20 -7.23 0.00 -20.75 -11.90 **
ot2 5.11 2.08 499.54 2.46 0.01 1.04 9.18 *
ot3 -0.20 2.01 524.30 -0.10 0.92 -4.14 3.74 -
age -0.51 0.02 34143.88 -23.66 0.00 -0.55 -0.46 **
ot1:age 0.10 0.09 12116.38 1.10 0.27 -0.08 0.27 -
ot2:age 0.16 0.09 7873.74 1.79 0.07 -0.01 0.33 .
ot3:age -0.43 0.09 6313.50 -4.96 0.00 -0.60 -0.26 **
Figure 6: fitted curve of the model with the prediction interval for each age group

Figure 6: fitted curve of the model with the prediction interval for each age group

Effect of Gender and Likelihood Group

Gender modulated the effect of age and of ot1. Males and females of different likelihood risks show different developmental trajectories:

Table 15: Estimates of fixed effects predicting the proportional looking time to the face, with the 95% confidence intervals
term Estimate st.error df t.value p.value CI_2.5 CI_97.5 Sign
(Intercept) 46.36 2.15 18.69 21.54 0.00 42.14 50.58 **
ot1 -23.50 3.36 425.93 -6.98 0.00 -30.09 -16.90 **
ot2 5.71 3.08 511.91 1.85 0.06 -0.33 11.76 .
ot3 0.03 3.00 533.13 0.01 0.99 -5.86 5.91 -
age -0.41 0.03 34934.94 -13.29 0.00 -0.47 -0.35 **
sexM -0.74 1.84 169.79 -0.40 0.69 -4.35 2.87 -
ot1:age 0.36 0.13 14249.03 2.76 0.01 0.10 0.61 *
ot2:age 0.22 0.13 9528.16 1.73 0.08 -0.03 0.47 .
ot3:age -0.51 0.13 7992.77 -3.99 0.00 -0.76 -0.26 **
ot1:sexM 13.13 4.54 424.77 2.89 0.00 4.23 22.03 *
ot2:sexM -1.05 4.16 500.56 -0.25 0.80 -9.20 7.10 -
ot3:sexM -0.50 4.05 516.23 -0.12 0.90 -8.45 7.44 -
age:sexM -0.18 0.04 34014.34 -4.16 0.00 -0.26 -0.09 **
ot1:age:sexM -0.49 0.18 12285.60 -2.73 0.01 -0.84 -0.14 *
ot2:age:sexM -0.12 0.18 7920.75 -0.70 0.48 -0.47 0.22 -
ot3:age:sexM 0.15 0.17 6553.03 0.86 0.39 -0.19 0.49 -

Table 16: Estimates of fixed effects predicting the proportional looking time to the face, with the 95% confidence intervals
term Estimate st.error df t.value p.value CI_2.5 CI_97.5 Sign
(Intercept) 45.94 3.44 83.00 13.36 0.00 39.20 52.68 **
ot1 -18.93 6.96 320.45 -2.72 0.01 -32.58 -5.29 *
ot2 0.97 6.34 384.06 0.15 0.88 -11.46 13.40 -
ot3 0.18 6.18 403.85 0.03 0.98 -11.93 12.28 -
age -0.29 0.06 36538.70 -4.89 0.00 -0.41 -0.18 **
sexM -0.65 3.86 150.96 -0.17 0.87 -8.22 6.92 -
ASD1 0.80 3.38 150.70 0.24 0.81 -5.83 7.43 -
ot1:age 0.26 0.25 17341.07 1.04 0.30 -0.23 0.76 -
ot2:age 0.30 0.25 10912.02 1.21 0.23 -0.19 0.79 -
ot3:age -0.49 0.25 9059.52 -1.98 0.05 -0.98 -0.01 *
ot1:sexM 14.48 9.07 339.95 1.60 0.11 -3.30 32.25 -
ot2:sexM 4.42 8.27 405.91 0.53 0.59 -11.79 20.64 -
ot3:sexM 5.12 8.07 426.10 0.64 0.53 -10.69 20.93 -
age:sexM -0.39 0.08 35862.26 -4.72 0.00 -0.55 -0.23 **
ot1:ASD1 -5.91 7.95 344.43 -0.74 0.46 -21.49 9.68 -
ot2:ASD1 6.11 7.27 413.05 0.84 0.40 -8.13 20.35 -
ot3:ASD1 -0.18 7.08 433.88 -0.03 0.98 -14.06 13.70 -
age:ASD1 -0.17 0.07 36006.04 -2.37 0.02 -0.30 -0.03 *
sexM:ASD1 -0.49 4.40 155.55 -0.11 0.91 -9.11 8.13 -
ot1:age:sexM -0.47 0.34 15961.03 -1.38 0.17 -1.15 0.20 -
ot2:age:sexM -0.13 0.34 9828.08 -0.38 0.71 -0.79 0.54 -
ot3:age:sexM -0.11 0.34 8098.54 -0.32 0.75 -0.77 0.55 -
ot1:age:ASD1 0.13 0.30 16012.18 0.44 0.66 -0.45 0.71 -
ot2:age:ASD1 -0.11 0.29 10382.67 -0.37 0.71 -0.68 0.46 -
ot3:age:ASD1 -0.02 0.29 8712.75 -0.07 0.94 -0.59 0.55 -
ot1:sexM:ASD1 -2.19 10.48 360.77 -0.21 0.83 -22.72 18.35 -
ot2:sexM:ASD1 -7.07 9.58 428.68 -0.74 0.46 -25.85 11.72 -
ot3:sexM:ASD1 -7.86 9.35 448.33 -0.84 0.40 -26.18 10.46 -
age:sexM:ASD1 0.28 0.10 35235.58 2.97 0.00 0.10 0.47 *
ot1:age:sexM:ASD1 -0.01 0.40 14322.14 -0.02 0.98 -0.80 0.78 -
ot2:age:sexM:ASD1 0.01 0.40 9037.48 0.02 0.98 -0.77 0.79 -
ot3:age:sexM:ASD1 0.36 0.40 7513.81 0.91 0.36 -0.42 1.13 -

Car

This models shows that the proportional looking time to the car tends to increase with age with a S-shaped profile (defined by the term ot3). Notably, infants with high-likelihood of ASD show a higher increase of the average looking time with age (age:ASD1). The trend is shown in Figure 7.

Table 17: Estimates of fixed effects predicting the proportional looking time to the car, with the 95% confidence intervals
term Estimate st.error df t.value p.value CI_2.5 CI_97.5 Sign
(Intercept) 12.44 1.80 53.64 6.93 0.00 8.92 15.96 **
ot1 1.35 3.18 419.94 0.42 0.67 -4.90 7.59 -
ot2 -1.80 3.10 418.70 -0.58 0.56 -7.87 4.27 -
ot3 0.09 2.96 564.51 0.03 0.98 -5.72 5.89 -
age 0.20 0.03 35816.37 6.55 0.00 0.14 0.26 **
ASD1 -2.27 1.69 162.81 -1.34 0.18 -5.58 1.04 -
ot1:age -0.17 0.13 11102.81 -1.33 0.18 -0.42 0.08 -
ot2:age -0.04 0.13 10486.83 -0.30 0.76 -0.29 0.21 -
ot3:age 0.11 0.13 8553.25 0.88 0.38 -0.14 0.36 -
ot1:ASD1 0.44 3.70 442.17 0.12 0.90 -6.81 7.69 -
ot2:ASD1 2.73 3.60 439.26 0.76 0.45 -4.33 9.78 -
ot3:ASD1 2.02 3.45 591.19 0.59 0.56 -4.73 8.78 -
age:ASD1 0.07 0.04 35274.96 1.96 0.05 0.00 0.14 *
ot1:age:ASD1 0.10 0.15 10212.23 0.69 0.49 -0.19 0.40 -
ot2:age:ASD1 -0.18 0.15 9484.37 -1.17 0.24 -0.47 0.12 -
ot3:age:ASD1 -0.05 0.15 8141.11 -0.35 0.73 -0.34 0.24 -
Figure 7: fitted curve of the model with the prediction interval for each age group

Figure 7: fitted curve of the model with the prediction interval for each age group

Individual Scores and Standard Errors

Let us consider the reliability of the coefficients reported in the previous section. In a mixed model, the coefficients constitute an average of the individual coefficients, whose deviation is estimated by the random effect. The model considers the reliability of the individual coefficient too, and weights it when including it in the coefficient calculation. The weight is highly dependent on the number of observations available for the individual - less observation, less reliability, less weight.

In the below plots, we visualise the individual coefficients, corresponding to the sum of the fixed effect and the individual variance component (as specified in the random effect). The error bars represent the standard errors, calculated as the the joint MSE of the summated variances of the fixed and the random effect. The first plot nicely illustrates how standard errors of the individual coefficients of the proportional looking time to the face increases for participants with fewer observations. In this model, we observe at least 50% of individual coefficient with a standard error above the average standard error.

The second plot shows an partially different situation. Even though we still observe a pattern in the distribution of errors, they are more prominently above the average. Crucially, the model estimating the proportional looking time to the car is more complex and includes 1 more regressors, corresponding to the risk group. A more complex model, while it reduces the bias due to the assumptions of a certain function, it substantially increases the variance of the model. In other words, the model reacts to small fluctuations in the data and may be prone to overfitting. Therefore, the conclusions of this model must be taken with care. The precision of this model may be improved by adding a validation set.

Fig. 8: Individual estimates and standard errors of the Intercept for the proportional looking time to the face and the car, on the numerosity of the observations per infant. The horizontal lines represent the average Intercept estimate + 95% CI

Fig. 8: Individual estimates and standard errors of the Intercept for the proportional looking time to the face and the car, on the numerosity of the observations per infant. The horizontal lines represent the average Intercept estimate + 95% CI